in revisiting the compact proofs of model performance and seeing progress in recent work I think I should evaluate how well this line of thought connects with my thoughts in compact proofs you are mainly looking for ways to wholly explain how some network (or less ambitiously, part of a network) generalizes by compressing all of its computations into an explanation. one approach that fits this idea is using SAEs or something similar that essentially contains all the training datapoints, and decomposes into linear combinations of features. with that, you can basically assume i.i.d over the features of the network you want to explain as behaviors and estimate. you can also replace parts of the model computation with SAE atoms and directly measure the loss recovered on the explanation deep causal transcoders overall do not take this approach but nonetheless produces very efficiently compressible explanations for model behaviors, somewhat. if you train an SAE over the activations of a model on 200 data points or so, you are much more likely to cover far fewer generalizations, so DCTs are probably more likely to give better compressing explanations for particular behaviors, and also efficiently find nonlinear interactions. but they aren't inherently probabilistic models of features, you don't get the added benefit of being able to measure feature salience against your training distribution (without having to do a different sampling process altogether); and ofc, they are also partial, you still somewhat get what you measure. with this in mind, how much should you rely on these as explanations? i have not tried training a DCT similar to an SAE but in the limit i expect that it would be functionally like an SAE. DCTs seem like a significant Pareto improvement on data efficiency that makes it simpler to produce explanations about hard-to-specify behaviors. i think this contrasts pretty heavily with the broader goal of mechinterp to form whole explanations for model generalizations; a standard explanation from DCTs could be very easy to overturn with new evidence despite this, i think given the generalization strengths of this approach, it should be worth trying to reduce and simplify DCTs as explanations, while minifying sampling, as this only grants more confidence in how features are used in the model, as factors map [non-linearly and can likely be made independent modules](https://docs.google.com/document/d/1hhR3_fQnUsWs3iv3B820SLxIsA4Qds4vHn1rJEXIBjs/?tab=t.0#heading=h.rzuxovo68xws). given there actually are independent modules, how should one go about using this mechanistic structure to make better probability estimates, supposing you don't just skip past the problem, or assume some unfounded constant uniform over all events, or extensively sample the training set? one method I had in mind was an activation space/logit lens metric for [surprisal-surprisal](https://www.lesswrong.com/posts/Kjo64rSWkFfc3sre5/detecting-out-of-distribution-text-with-surprisal-and) when steering or ablating $S$ features. but i think i was missing a lot of complexity, because the original method computes surprisal and entropy over prefilled tokens and expected token prediction, and basically there is no such distribution for model features that we can read off for similar calculations. in effect you probably would just be taking the kl distance from the steered logit lens distribution to the base distribution and this seems too simplistic and local another method is, supposing $I_l(S)$ where $S = [v_1, ..., v_n]$, you could measure the number of standard deviations of an attribution from if $S$ consists of random unit vectors... ok, that doesn't seem like a great measurement since random vecs are ~never privileged here. so what about the same metric using permutations of SAE vectors at the source layer, or crosscoder vectors on the source and target layers? seems like a good baseline here; this is roughly what i had in mind since [here](<./circuit utility.md>) although I didn't previously know how to measure this counterfactual. this also seems like a good way to verify that DCTs can be factorized and reduced in practice. one potential issue is that crosscoder features may just have high variance because they're pathological in the context of DCT features. in practice this turned out to be the case, in addition to the attribution method intended to measure interactions between features that are already conditioned to work in this expm1/causality basis. training crosscoders with gated expm1 activations didn't work either. such latents were almost completely orthogonal to features in $V$, pca plotting also didn't discern anything useful. translating crosscoder latents into DCT space seems unworkable and/or useless. dunno what to do from here. perhaps it'll be more worthwhile to try understanding what is so special about high attribution-scoring DCT circuits at all. but if we want a better 'generalization' statistic, measuring stddevs in attribution from features located other bases (like crosscoders, transcoders, or SAEs) is not going to work without modifying the other methods to work almost exactly like DCTs. maybe there's more to learn from [here](https://www.lesswrong.com/posts/uDrsffSLzWD6cDnTt/evidence-for-feature-specific-error-correction-in-llms-1#Superellipse_exponents_across_direction_types) and [here](https://www.lesswrong.com/posts/LajDyGyiyX8DNNsuF/interim-research-report-activation-plateaus-and-sensitive-1), or in the literature for latent/ontology translation (which eventually works out to some simple ml trick). well the 'SOTA' for latent translation is not made for any computational units, so forget that. a different idea is to test limitations of sparse coding for reconstructing sailent DCT circuits (after proving they are useful generalizations? not sure), then you can make some sparsity-related assumptions about what it would take for an ideal crosscoder to match the generalization of a functionally similar DCT and use that as a saliency measure... here i also got relatively boring results, like yeah, obviously sparse coding is able to reconstruct the input output features, and no, it can barely reconstruct input features to reliably predict some output direction, and so you would get pathological amounts of variance again. i also threw 5.6 Sol at producing better analytic constructions for attribution that did not depend on expm1; i got some slop where it tried implementing a separate attribution function based on dot products of $U,V$ with themselves, and another function that was reportedly better on 'held-out samples', but it required another neural net to fit to the DCT using transformer activations, and it didn't look particularly useful for me anyways. it also produced some negative results for $I_j(S)$ on some form of held out cases with the transformer, but im unsure they were relevant to what i'm doing, because in another session i was able to get Sol to red-team $I_j(S)$ and it was able to show generalization to finding non-linear feature interactions in transformers that are better at steering between generalizations than individual features. this seems to validate a lot about the structure of the DCT which seems good. but i remain confused on how to measure counterfactuals using features located in other bases just based on the parameters. otoh, i did another red-team and found the function does not identify circuits that certainly interact in more meaningfully ways in the transformer compared to individual features, so really we *don't* have explanations over the transformer. so in terms of the actual title of this post: "dcts as well-compressed partial explanations," i think it seems wrong in the sense of the terms there, or it also doesn't say much lol. sparse features are well compressed, features that encode a lot of downstream causality might encode a lot of *computational meaning,* but they are not particularly compressed. like sure, an explanation encoded as a DCT *circuit* is probably easier to compress thanks to dependence on the idealized MLP activation function, $e^x-1$, compared to ReLU/TopK/whatever SAEs use for non-linearly interacting features. but individual features, not particularly. still though, based on features interacting through expm1, and also encoding more useful computation, i think its fair to say they are probably 'well-compressed' explanations. and of course they are partial explanations because DCTs in expectation are extremely poor for decomposing an entire training set into circuits the biggest thing right now is how should we compare dct circuits as explanations/existence proof for a generalization, compared to equivalent circuits implemented in some other basis, like sparse coding, and how 'surprising' the generalization is in the context of what sparse coding can realistically do. even with $I_j(S)$ not actually being a useful attribution function, there is certainly still non-linear interactions between features in the DCT, and DCTs probably model things in at least partially causal ways. i had written a longer pg here about why i expect DCTs to not work well for this problem at all before finding a better [approach](https://www.lesswrong.com/posts/8iG7orf6q8hNHT4Xh/ajskateboarder-s-shortform?commentId=vyDRwt434Y9pz9Jgm) which better decomposes deep bilinear transformer slices into feature circuits (and as i found later, plain swiglu transformers). it necessarily uses gradient ascent to recover asymmetric causally important features so it doesn't have the same guarantees as closed-form orth-ALS but it converges close enough to the original recovery guarantees, that it seems basically good enough to me, i guess